Image capturing method, apparatus, and storage medium

ABSTRACT

A method and an apparatus for image capturing, and a storage medium can be applied to a terminal device. When it is determined that the terminal is in an augmented reality image capturing scene, augmented reality environment information of the augmented reality image capturing scene is determined. When an image capturing instruction is received, an image is captured and augmented reality environment information corresponding to the image is recorded to obtain a target image carrying the augmented reality environment information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No.202010101946.2, filed on Feb. 19, 2020, the disclosure of which ishereby incorporated by reference in its entirety.

BACKGROUND

Augmented Reality (AR) technology is to increase a perception of a userto the real world through information provided by a computer system,apply virtual information to the real world, and superimpose virtualobjects, scenes, or systematic prompt information generated by thecomputer system to a real scene, to augment the reality.

SUMMARY

The present disclosure generally relates to the field of augmentedreality technology, and more specifically, to an image capturing method,apparatus, and storage medium.

According to embodiments of the present disclosure, there is provided amethod for image capturing, applicable to a terminal device. The methodcan include: in response to determining that the terminal in anaugmented reality image capturing scene, augmented reality environmentinformation of the augmented reality image capturing is determined. Inresponse to receiving an image capturing instruction, an image iscaptured and the augmented reality environment information correspondingto the image is recorded to obtain a target image carrying the augmentedreality environment information.

According to the present disclosure, there is provided a device forimage capturing, including a memory and a processor. The memory isconfigured to store instructions. The processor is configured to callthe instructions to execute the method for image capturing according tothe first aspect or of any example of the first aspect.

According to the present disclosure, there is provided a non-transitorycomputer-readable storage medium, having computer-executableinstructions stored thereon. When the instructions are executed by theprocessor, a method for image capturing according to the first aspect orof any example of the first aspect is executed.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthe disclosure, illustrate embodiments of the present disclosure andserve to explain principles of the present disclosure together with thespecification.

FIG. 1 is a flowchart illustrating a method for image capturingaccording to some embodiments.

FIG. 2 is a semantic diagram illustrating AR environmental informationaccording to some embodiments.

FIG. 3 is a flowchart illustrating a method for image capturingaccording to some embodiments.

FIG. 4 is a flowchart illustrating a method for image capturingaccording to some embodiments.

FIG. 5 is a schematic diagram illustrating performing an augmentedreality operation on a target picture some embodiments.

FIG. 6 is a flowchart illustrating a method for image capturingaccording to some embodiments.

FIG. 7 is a flowchart illustrating a method for image capturingaccording to some embodiments.

FIG. 8 is a block diagram illustrating an apparatus for image capturingaccording to some embodiments.

FIG. 9 is a block diagram illustrating a device according to someembodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to example embodiments, examples ofwhich are illustrated in the accompanying drawings. The followingdescription refers to the accompanying drawings in which the samenumbers in different drawings represent the same or similar elementsunless otherwise represented. The implementations set forth in thefollowing description of example embodiments do not represent allimplementations consistent with the present disclosure. Instead, theyare merely examples of apparatuses and methods consistent with aspectsrelated to the present disclosure as recited in the appended claims.

Users may not be satisfied with taking a photo of the real scene, butmay want to merge virtual contents into their photos to achieve theaugmented reality and make the photos look more beautiful or attractive.Augmented reality technologies can meet requirements of the users byoverlaying augmented content, such as the virtual content, onto theimage captured by the camera to obtain an AR image.

To overlay the virtual content onto the shot image, an image capturingapplication may have to be enabled. In addition, in the AR imagegenerated by overlaying the virtual content onto the captured image, theAR effect realized by the virtual content is often fixed with pixels ofthe image, which cannot be modified or edited.

Various embodiments of the present disclosure can provide a method forimage capturing, which can modify and edit the AR effect of the capturedimage without enabling the image capturing application.

Various embodiments of the present disclosure can be applied to anapplication scene of image capturing by using a camera provided on aterminal. In the example embodiments described below, the terminal mayalso be referred to as a smart terminal device. The terminal may be amobile terminal, such as user equipment (UE), a mobile station (MS),etc. The terminal is a device that provides voice and/or data connectionto the user, or a chip disposed within the device; for example, ahandheld device, a vehicle-mounted device, etc., with wirelessconnectivity. Examples of the terminal may include a mobile phone, atablet computer, a notebook computer, a palm computer, a mobile Internetdevice (MID), a wearable device, a virtual reality (VR) device, anaugmented reality (AR) device, a wireless terminal in a field ofindustrial control, a wireless terminal in a field of unmanned driving,a wireless terminal in a field of remote surgery, a wireless terminal ina smart grid, a wireless terminal in a field of transportation safety, awireless terminal in a smart city, a wireless terminal in a smart home,etc.

FIG. 1 is a flowchart illustrating a method for image capturingaccording to some embodiments. As illustrated in FIG. 1 , the method isapplied in a terminal and includes the following.

In block S11, it is determined that the terminal is in the augmentedreality (AR) image capturing scene, and the augmented realityenvironment information of the augmented reality image capturing sceneis determined.

In the present disclosure, for example, when an instruction for enablingan AR image capturing mode is received, the AR image capturing mode ofthe image capturing apparatus is enabled, to determine that the terminalis in the AR image capturing scene.

In order to establish robust AR environment information of the AR imagecapturing scene, the environment information of the image capturingscene may be scanned by the image capturing apparatus. The robust ARenvironment information of the image capturing scene may be obtainedafter the image capturing apparatus scans the environment information ofthe image capturing scene. The AR environment information scanned by theimage capturing apparatus may be, for example, one or more of planeinformation, light source information, and environment depth informationof the image capturing scene.

The plane information of the image capturing scene may include, forexample, the number of planes, position and size of each plane, etc. Thelight source information of the image capturing scene may include, forexample, the number of light sources, the position of the light source,and the like. The environment depth information may be, for example,three-dimensional positions and sizes of the image capturingenvironment.

FIG. 2 is a schematic diagram illustrating the scanning of theenvironment information of the image capturing scene by the imagecapturing apparatus to obtain the robust AR environment information ofthe image capturing scene according to some embodiments.

As illustrated in FIG. 2 , after the environment information of theimage capturing scene is scanned by the image capturing apparatus,available plane information may include the position and size of a planeregion of the ground having grid lines that are interlaced to each otherand the position and size of a plane region of a desktop having gridlines that are interlaced with each other. In addition, after theenvironment information of the image capturing scene is scanned by theimage capturing apparatus, the environment depth information isobtained, including information of three-dimensional positions and sizesof the image capturing environment generated from discrete points markedin an image and the plane information.

In block S12, when an image capturing instruction is received, an imageis captured and the augmented reality environment informationcorresponding to the image is recorded to obtain a target image carryingthe augmented reality environment information.

In some embodiments of the present disclosure, after the robust ARenvironment information is established based on the AR image capturingscene, the image may be captured according to the received imagecapturing instruction and the AR environment information correspondingto the image may be recorded, to obtain the image carrying the ARenvironment information.

In some embodiments, it is determined that the terminal is in the ARimage capturing scene, and the AR environment information of the ARimage capturing scene is determined. When the image capturinginstruction is received, the image is captured and the AR environmentinformation corresponding to the image is recorded to obtain the imagecarrying the AR environment information. In addition, according to theAR environment information carried in the captured image, modificationand edition of the AR effect on the captured image may be realizedwithout enabling the image capturing application.

In some embodiments of the present disclosure, the received imagecapturing instruction may be an instruction for capturing the image oran instruction for capturing a video. The present disclosure will bedescribed in detail below with reference to practical applications, inwhich a case for capturing a picture based on the image capturinginstruction and a case for capturing a video based on the imagecapturing instruction may be described respectively.

The present disclosure will be described in detail below with respect toa case that the picture is captured by applying the method according tothe present disclosure.

FIG. 3 is a flowchart illustrating a method for image capturingaccording to some embodiments. As illustrated in FIG. 3 , the method isapplicable in the terminal and may include blocks S31 and S32. The blockS31 is similar to the block S11 of FIG. 1 , which is not elaboratedhere.

In block S32, when the instruction for capturing a picture is received,a preview image is acquired, and the augmented reality environmentinformation for generating the preview image is recorded using anaugmented reality algorithm. The preview image is associated with theaugmented reality environment information for generating the previewimage to obtain the target image carrying the augmented realityenvironment information.

In some embodiments of the present disclosure, in order to prevent thecase that the recorded AR environment information is inaccurate due to asize inconsistency between the preview image and a captured image, thepresent disclosure may set in advance a same field of view (FOV) foracquiring the preview image and for capturing the image. Therefore, whenthe instruction for capturing the picture is received, the FOV foracquiring the preview image and the FOV for capturing the image arealways consistent with each other. Therefore, the accuracy of therecorded AR environment information of the target image should beimproved.

In an implementation, after the AR environment information for acquiringeach preview image is recorded using the AR algorithm, each previewimage may be associated with the AR environment information foracquiring the preview image to obtain the target image carrying the ARenvironment information.

After the AR environment information for acquiring each preview image isrecorded using the AR algorithm, each preview image may be associatedwith one or more pieces of AR environment information for acquiring therespective preview image in a form of key-value pair.

For example, the AR environment information of the image includes theplane information, the light source information, and the environmentdepth information of the image. After the AR environment information ofeach preview image is recorded using the AR algorithm, the preview imageis associated with the plane information of the preview image, the lightsource information of the preview image and the environment depthinformation of the preview image in the form of key-value pair, suchthat the preview image and several pieces of AR environment informationof the preview image form a set of AR environment information.

In some embodiments, by determining that the terminal is in the AR imagecapturing scene, the AR environment information of the AR imagecapturing scene is determined. When the instruction for capturing apicture is received, the AR environment information for acquiring eachpreview image is recorded using the AR algorithm, and each preview imageis associated with the AR environment information for acquiring therespective preview image to obtain the target image carrying the ARenvironment information. Therefore, according to the AR environmentinformation carried in the target image, modification and edition on theAR effect of the target image may be implemented, thereby enriching anapplication scene of AR.

FIG. 4 is a flowchart illustrating a method for image capturingaccording to some embodiments. As illustrated in FIG. 4 , the method isapplicable to a terminal and may include blocks S41, S42, S43, and S44.The blocks S41 and S42 are similar to the blocks S31 and S32 of FIG. 3 ,which are not elaborated here.

In block S43, according to a file storage type of the target image, theaugmented reality environment information corresponding to the targetimage is packaged into a file of the target image to obtain a targetpicture carrying the augmented reality environment information.

In some embodiments of the present disclosure, according to the filestorage type of the target image, the AR environment informationcorresponding to the target image may be packaged into the file of thetarget image to obtain the target picture carrying the AR environmentinformation.

For example, the file storage type of the target image is a JPEG format.A length of several AR environment information, i.e., a set of ARenvironment information, of the target image is written into metadata(such as xmp) of image file of the JPEG format and the set of ARenvironment information of the target image is attached to the end ofthe image file of the JPEG format.

As another example, the file storage type of the target image is a HEIFformat. Since the HEIF format allows customized data to be written intothe image file as an independent data block, the data can be customized,and the set of AR environment information of the target image may bewritten into the image file of the HEIF format as independent data.

In block S44, when an instruction for performing augmented reality onthe target picture is received, an augmented reality processing isperformed on the target picture according to the augmented realityenvironment information of the target picture.

In some embodiments of the present disclosure, the instruction forperforming the AR on the target picture may be generated based on theuser's selection on AR processing instructions when the image is editedby the user. The instruction for performing the AR on the target picturemay include overlapping an AR image onto the target picture, replacingthe AR effect of the target picture, and overlaying a sound effect onthe target picture, but the present disclosure is not limited thereto.

In an implementation, after the augmented reality environmentinformation corresponding to the target image is packaged into the fileof the target image based on the file storage type of the target imageto obtain the target picture carrying the augmented reality environmentinformation, and when the instruction for performing the AR on thetarget picture is received, the target image is decoded to obtain the ARenvironment information of the target picture. The AR is performed onthe target picture based on the AR environment information of the targetpicture and a preset AR processing manner.

For example, the file storage type of the target image is the JPEGformat. The length of the several AR environment information, i.e., theset of AR environment information, of the image picture is written intothe metadata (e.g. xmp) of the image file of the JPEG format, and thedata of the set of AR environment information of the target image isattached to the end of the image file of the JPEG format. After aninstruction for overlapping an AR image onto the target picture isreceived, the target picture is decoded. In decoding the target picture,in addition to generally decoding the image file of the JPEG format, thedata of the set of AR environment information of the target picture atthe end of the image file of the JPEG format is decoded. Since thebeginning and the length of the set of AR environment information arerecorded in a header of the image file of the JPEG format when the imagefile is generated and it is unaware of the set of AR environmentinformation of the target picture attached to the end of the image file,it requires to obtain the length from the metadata, such as xmp, throughdecoding, such that the image file is decoded from a position of the setof AR environment information that is calculated based on the lengthstarting from an end point of the image file, to obtain the set of ARenvironment information of the target picture. The AR processing isperformed on the target picture based on the AR environment informationof the target picture and the instruction for performing the AR on thetarget picture.

FIG. 5 is a schematic diagram illustrating the augmented realityprocessing on the target picture according to the instruction from someembodiments.

In FIG. 5 , when the instruction for performing the augmented realityprocessing on the target picture is received, for example aftertriggering the touch key “AR” as illustrated in FIG. 5 , the AR imagemay be overlapped onto the target image within the region formed by thegrid lines of the AR plane based on the AR environment information ofthe picture.

In some embodiments, after the AR environment information of the targetimage is packaged into the file of the target image based on the filestorage type of the target image to obtain the target picture carryingthe AR environment information, and when the instruction for performingthe AR on the target picture is received, the AR processing may beperformed on the target picture based on the AR environment informationof the target picture, thereby allowing the target picture to bere-edited secondarily, enriching the application scene of AR, andimproving user experience.

The present disclosure will be described in detail below with respect tothe case that the video is captured by applying the method according tothe present disclosure.

FIG. 6 is a flowchart illustrating a method for image capturingaccording to some embodiments. As illustrated in FIG. 6 , the method maybe applicable to a terminal and may include blocks S61 and S62. Theblock S61 is similar to the block S11 of FIG. 1 , which is notelaborated here.

In block S62, when the instruction for image capturing a video isreceived, during image capturing the video, the augmented realityenvironment information corresponding to each video frame of the shotvideo is sequentially recorded using the augmented reality algorithm.Based on a display timestamp of each video frame, each video frame ofthe shot video is associated with the augmented reality environmentinformation corresponding to respective video frame, to obtain a targetvideo carrying the augmented reality environment information.

In some embodiments of the present disclosure, when the instruction forimage capturing a video is received, during image capturing the video,the augmented reality environment information corresponding to eachvideo frame of the shot video is recorded in sequence according to thetime sequence of the shot video.

After the augmented reality environment information corresponding toeach video frame of the shot video is obtained, each video frame may beassociated with one or more pieces of augmented reality environmentinformation of respective video frame in a form of key-value pair, suchthat the video frame and several AR environment information of the videoframe form a set of environment information in various dimensions.

According to the display timestamp of each video frame, each video frameof the shot video is associated with the set of several AR environmentinformation of the respective video frame to obtain the target videocarrying the augmented reality environment information.

In some embodiments of the present disclosure, the augmented realityenvironment information of the augmented reality image capturing sceneis determined by determining that the terminal is in the augmentedreality image capturing scene. When the instruction for capturing avideo is received, the augmented reality environment informationcorresponding to each video frame of the shot video is recorded usingthe augmented reality algorithm. Based on a display timestamp of eachvideo frame, each video frame of the shot video is associated with theaugmented reality environment information corresponding to therespective video frame to obtain the target video carrying the augmentedreality environment information. Therefore, according to the augmentedreality environment information carried in the target video,modification and edition on the AR effect of the target video may beimplemented.

FIG. 7 is a flowchart illustrating a method for image capturing withaugmented reality according to some embodiments. As illustrated in FIG.7 , the method may be applicable to a terminal and may include blocksS71, S72, S73, and S74. The blocks S71 and S72 are similar to the blocksS61 and S62 of FIG. 6 , which are not elaborated here.

In block S73, according to the file storage type of the target video,the augmented reality environment information for capturing the targetvideo is packaged into the file of the target video to obtain a targetvideo file carrying the augmented reality environment information.

In some embodiments of the present disclosure, based on the file storagetype of the target video, the AR environment information of the targetvideo may be packaged into the file of the target video to obtain thetarget video file carrying the AR environment information.

For example, the file storage type of the target video is a mkv format.The mkv format allows to customize data structure. For example, amulti-track video file and a multi-caption video file may be customizedin the video file of the mkv format. A video track that is displayedover time may be generated from a “frame,” i.e., the AR environmentinformation of each frame (that is, the set of the AR environmentinformation), together with the display timestamp corresponding to thisframe, and may be written into the video file of the mkv format.

In block S74, when an instruction for performing augmented reality onthe target video file is received, the augmented reality processing isperformed on the target video file according to the augmented realityenvironment information of the target video.

In some embodiments of the present disclosure, for example, theinstruction for performing the AR on the target video file may bereceived based on the user's selection on the AR processing instructionswhen the user edits the video. The instruction for performing AR on thetarget video file may include overlapping an AR image onto the targetvideo file, replacing the AR image of the target video file, overlappinga sound effect onto the target video file.

In an implementation, after the AR environment information of the targetvideo is packaged into the file of the target video according to thefile storage type of the target video, to obtain the target video filecarrying the augmented reality environment information, and when theinstruction for performing the AR on the target video file is received,the AR processing may be performed on the target video file based on theAR environment information of the target video file and a preset ARprocessing manner.

For example, when an instruction for overlapping the AR sound effectonto the target video file is received, target video file is received,the AR sound effect is overlaid onto the target video file based on theAR environment information of the target video and a preset AR soundeffect.

In examples of the present disclosure, after the AR environmentinformation of the target video is packaged into the file of the targetvideo based on the file storage type of the target video, to obtain thetarget video file carrying the AR environment information and when theinstruction for performing AR on the target video file is received, theAR processing may be performed on the target video file according to theAR environment information of the target video file, thereby allowingthe target video to be re-edited and improving user experience.

Based on the same inventive concept, the present disclosure furtherprovides an apparatus for image capturing with augmented reality.

It may be understood that, in order to implement the above functions,the apparatus provided in embodiments of the present disclosure mayinclude a hardware structure and/or a software module for performingeach function. Embodiments of the present disclosure can be implementedin hardware or a combination of hardware and computer software, incombination with the exemplary elements and algorithm steps disclosed inthe disclosed embodiments. Whether to perform the function in hardwareor by hardware driven by the computer software depends upon theparticular application and design constraints of the solution. Thoseordinary in the art may implement the described functions in varyingways depending on each particular application, but such implementationsshould not be interpreted as a departure from the scope of theembodiments of the present disclosure.

FIG. 8 is a block diagram illustrating an apparatus for image capturingaccording to some embodiments. As illustrated in FIG. 8 , the apparatusmay include a determining unit 101 and a processing unit 102.

The determining unit 101 may be configured to, in response todetermining that the terminal is in the augmented reality imagecapturing scene, determine augmented reality environment information ofthe augmented reality image capturing scene;

The processing unit 102 may be configured to, in response to receivingan image capturing instruction, capture an image and record theaugmented reality environment information corresponding to the image toobtain a target image carrying the augmented reality environmentinformation.

In some embodiments, the augmented reality environment information mayinclude one or more of plane information, light source information, andenvironment depth information of the image capturing scene.

In some embodiments, the image capturing instruction is an instructionfor capturing an image. The processing unit 102 is configured to capturethe image and record the augmented reality environment informationcorresponding to the image to obtain the target image carrying theaugmented reality environment information by the following. A previewimage is acquired. The augmented reality environment informationcorresponding to the preview image is recorded using an augmentedreality algorithm. Each preview image is associated with the augmentedreality environment information of the respective preview image, toobtain the target image carrying the augmented reality environmentinformation.

In some embodiments, the image capturing instruction is an instructionfor image capturing a video. The processing unit 102 is configured tocapture the image and record the augmented reality environmentinformation corresponding to the image to obtain the target imagecarrying the augmented reality environment information by the following.When capturing the video, the augmented reality environment informationof each video frame of the shot video is sequentially recorded using theaugmented reality algorithm. Based on display timestamp of each videoframe, each video frame of the shot video is associated with theaugmented reality environment information of the respective video frameto obtain the target video carrying the augmented reality environmentinformation.

In some embodiments, the processing unit 102 is further configured to,based on the file storage type of the target image, package theaugmented reality environment information of the target image into thefile of the target image to obtain the target picture carrying theaugmented reality environment information. In addition, the processingunit 102 is further configured to, when receiving an instruction forperforming augmented reality on the target picture, perform theaugmented reality processing on the target picture according to theaugmented reality environment information of the target picture.

In some embodiments, the processing unit 102 is further configured to,package the augmented reality environment information of the targetvideo into the file of the target video based on the file storage typeof the target video, to obtain the target video file carrying theaugmented reality environment information. In addition, the processingunit 102 is further configured to, when receiving an instruction forperforming augmented reality on the target video file, perform theaugmented reality processing on the target video file according to theaugmented reality environment information of the target video file.

With regard to the apparatus in the above embodiment, the specific modein which each module performs the operation has been described in detailin the embodiments related to the method, and will not be described indetail here.

FIG. 9 is a block diagram illustrating an apparatus 900 for imagecapturing according to some embodiments. For example, the apparatus 900may be a mobile phone, a computer, a digital broadcast terminal, amessaging device, a game console, a tablet device, a medical device, anexercise device, a personal digital assistant, and the like.

As illustrated in FIG. 9 , the apparatus 900 may include one or more offollowing components: a processing component 902, a component memory904, a power supply component 906, a multimedia component 908, an audiocomponent 910, an input/output (I/O) interface 912, a sensor component914, and a communication component 916.

The processing component 902 generally controls the overall operation ofthe apparatus 900, such as operations associated with a display,telephone calls, data communications, camera operations, and recordingoperations. Further, the processing component 902 may include one ormore processors 920 to execute instructions to perform all or part ofthe steps of the method described above. Further, the processingcomponent 902 may include one or more modules to facilitate processingof interactions between processing component 902 and other components.For example, the processing component 902 may include a multimediamodule to facilitate the interaction between the multimedia component908 and the processing component 902.

The memory 904 is configured to store various types of data to supportoperations of the apparatus 900. Examples of such data includeinstructions for any application or method operating on the apparatus900, contact data, phonebook data, messages, pictures, videos, and thelike. The memory 904 may be implemented by any type of volatile ornon-volatile storage device or a combination thereof, such as s staticrandom access memory (SRAM), an electrically erasable programmable readonly memory (EEPROM), an erasable programmable read only memory (EPROM),a programmable read only memory (PROM), a read only memory (ROM), amagnetic memory, a flash memory, a disk or CD-ROM.

The power supply component 906 provides power to various components ofthe apparatus 900. The power supply component 906 may include a powermanagement system, one or more power supplies, and other componentsassociated with generating, managing, and distributing power suppliesfor the apparatus 900.

The multimedia component 908 includes a screen providing an outputinterface between the apparatus 900 and the user. In some embodiments,the screen may include a liquid crystal display (LCD) and a touch panel(TP). In some embodiments, an organic light-emitting diode (OLED)display can be employed.

If the screen includes the TP, the screen may be implemented as a touchscreen to receive input signals from the user. The TP includes one ormore touch sensors to sense touch, swipe, and gestures on the TP. Thetouch sensor may not only sense the boundary of a touch or slidingmotion, but also detect the duration and pressure related to the touchor slide operation. In some embodiments, the multimedia component 908includes a front camera and/or a rear camera. When the apparatus 900 isin an operation mode, such as an image capturing mode or a video mode,the front camera and/or the rear camera may receive external multimediadata. Each front camera and rear camera may be a fixed optical lenssystem or have a focal length and an optical zoom capability.

The audio component 910 is configured to output and/or input audiosignals. For example, the audio component 910 includes a microphone(MIC). When the apparatus 900 is in the operation mode, such as a callmode, a record mode, and a voice recognition mode, the MIC is configuredto receive external audio signals. The received audio signals may befurther stored in the memory 904 or sent via the communication component916. In some embodiments, the audio component 910 further includes aspeaker for outputting audio signals.

The I/O interface 912 provides an interface between the processingcomponent 902 and a peripheral interface module. The peripheralinterface module may be a keyboard, a click wheel, a button, and thelike. These buttons may include, but are not limited to: a home button,a volume button, a start button, and a lock button.

The sensor component 914 includes one or more sensors, configured toprovide various aspects of a state assessment for the apparatus 900. Forexample, the sensor component 914 may detect an open/closed state of theapparatus 900, and a relative positioning of the components, such as adisplay and a keypad of the apparatus 900. The sensor component 914 mayfurther detect a change in a position of the apparatus 900 or acomponent of the apparatus 900, the presence or absence of a contactbetween the user and the apparatus 900, the position or anacceleration/deceleration of the apparatus 900 and the temperaturechange of the apparatus 900. The sensor component 914 may include aproximity sensor configured to detect the presence of nearby objects inthe absence of any physical contact. The sensor component 914 mayfurther include an optical sensor, such as a CMOS or a CCD image sensor,for use in an imaging application. In some embodiments, the sensorcomponent 914 may also include an acceleration sensor, a gyroscopesensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 916 is configured to facilitate a wired or awireless communication between the apparatus 900 and other devices. Theapparatus 900 may access a wireless network based on communicationstandards, such as WI-FI, 2G, 3G, 4G, 5G, or a combination thereof. Inan exemplary embodiment, the communication component 916 receives abroadcast signal or broadcast-related information from an externalbroadcast management system via a broadcast channel. In someembodiments, the communication component 916 further includes a nearfield communication (NFC) module to facilitate a short-rangecommunication. For example, an NFC module may be implemented based on aradio frequency identification (RFID) technology, an infrared dataassociation (IrDA) technology, an ultra-wideband (UWB) technology, aBluetooth (BT) technology and other technologies.

In some embodiments, the apparatus 900 may be implemented by one or moreapplication specific integrated circuits (ASIC), digital signalprocessors (DSP), digital signal processing equipment (DSPD),programmable logic devices (PLD) and field programmable gate arrays(FPGA), controllers, microcontrollers, microprocessors and otherelectronic components, and configured to execute the above method.

In some embodiments, a non-temporary computer readable storage mediumincluding instructions is further provided, for example, the memory 904including instructions. The above instructions may be executed by theprocessor 920 of the apparatus 900 to complete the above method. Forexample, the non-temporary computer readable storage medium may be aROM, a random-access memory (RAM), a CD-ROM, a magnetic tape, a floppydisk, and an optical data storage equipment, and the like.

The various device components, modules, units, blocks, or portions mayhave modular configurations, or are composed of discrete components, butnonetheless can be referred to as “modules” in general. In other words,the “components,” modules,” “blocks,” “portions,” or “units” referred toherein may or may not be in modular forms, and these phrases may beinterchangeably used.

In the present disclosure, the terms “installed,” “connected,”“coupled,” “fixed” and the like shall be understood broadly, and can beeither a fixed connection or a detachable connection, or integrated,unless otherwise explicitly defined. These terms can refer to mechanicalor electrical connections, or both. Such connections can be directconnections or indirect connections through an intermediate medium.These terms can also refer to the internal connections or theinteractions between elements. The specific meanings of the above termsin the present disclosure can be understood by those of ordinary skillin the art on a case-by-case basis.

In the description of the present disclosure, the terms “oneembodiment,” “some embodiments,” “example,” “specific example,” or “someexamples,” and the like can indicate a specific feature described inconnection with the embodiment or example, a structure, a material orfeature included in at least one embodiment or example. In the presentdisclosure, the schematic representation of the above terms is notnecessarily directed to the same embodiment or example.

Moreover, the particular features, structures, materials, orcharacteristics described can be combined in a suitable manner in anyone or more embodiments or examples. In addition, various embodiments orexamples described in the specification, as well as features of variousembodiments or examples, can be combined and reorganized.

Implementations of the subject matter and the operations described inthis disclosure can be implemented in digital electronic circuitry, orin computer software, firmware, or hardware, including the structuresdisclosed herein and their structural equivalents, or in combinations ofone or more of them. Implementations of the subject matter described inthis disclosure can be implemented as one or more computer programs,i.e., one or more portions of computer program instructions, encoded onone or more computer storage medium for execution by, or to control theoperation of, data processing apparatus.

Alternatively, or in addition, the program instructions can be encodedon an artificially-generated propagated signal, e.g., amachine-generated electrical, optical, or electromagnetic signal, whichis generated to encode information for transmission to suitable receiverapparatus for execution by a data processing apparatus. A computerstorage medium can be, or be included in, a computer-readable storagedevice, a computer-readable storage substrate, a random or serial accessmemory array or device, or a combination of one or more of them.

Moreover, while a computer storage medium is not a propagated signal, acomputer storage medium can be a source or destination of computerprogram instructions encoded in an artificially-generated propagatedsignal. The computer storage medium can also be, or be included in, oneor more separate components or media (e.g., multiple CDs, disks, drives,or other storage devices). Accordingly, the computer storage medium canbe tangible.

The operations described in this disclosure can be implemented asoperations performed by a data processing apparatus on data stored onone or more computer-readable storage devices or received from othersources.

The devices in this disclosure can include special purpose logiccircuitry, e.g., an FPGA (field-programmable gate array), or an ASIC(application-specific integrated circuit). The device can also include,in addition to hardware, code that creates an execution environment forthe computer program in question, e.g., code that constitutes processorfirmware, a protocol stack, a database management system, an operatingsystem, a cross-platform runtime environment, a virtual machine, or acombination of one or more of them. The devices and executionenvironment can realize various different computing modelinfrastructures, such as web services, distributed computing, and gridcomputing infrastructures.

A computer program (also known as a program, software, softwareapplication, app, script, or code) can be written in any form ofprogramming language, including compiled or interpreted languages,declarative or procedural languages, and it can be deployed in any form,including as a stand-alone program or as a portion, component,subroutine, object, or other portion suitable for use in a computingenvironment. A computer program can, but need not, correspond to a filein a file system. A program can be stored in a portion of a file thatholds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more portions, sub-programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this disclosure can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA, or an ASIC.

Processors or processing circuits suitable for the execution of acomputer program include, by way of example, both general and specialpurpose microprocessors, and any one or more processors of any kind ofdigital computer. Generally, a processor will receive instructions anddata from a read-only memory, or a random-access memory, or both.Elements of a computer can include a processor configured to performactions in accordance with instructions and one or more memory devicesfor storing instructions and data.

Generally, a computer will also include, or be operatively coupled toreceive data from or transfer data to, or both, one or more mass storagedevices for storing data, e.g., magnetic, magneto-optical disks, oroptical disks. However, a computer need not have such devices. Moreover,a computer can be embedded in another device, e.g., a mobile telephone,a personal digital assistant (PDA), a mobile audio or video player, agame console, a Global Positioning System (GPS) receiver, or a portablestorage device (e.g., a universal serial bus (USB) flash drive), to namejust a few.

Devices suitable for storing computer program instructions and datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

To provide for interaction with a user, implementations of the subjectmatter described in this specification can be implemented with acomputer and/or a display device, e.g., a VR/AR device, a head-mountdisplay (HMD) device, a head-up display (HUD) device, smart eyewear(e.g., glasses), a CRT (cathode-ray tube), LCD (liquid-crystal display),OLED (organic light emitting diode), or any other monitor for displayinginformation to the user and a keyboard, a pointing device, e.g., amouse, trackball, etc., or a touch screen, touch pad, etc., by which theuser can provide input to the computer.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back-endcomponent, e.g., as a data server, or that includes a middlewarecomponent, e.g., an application server, or that includes a front-endcomponent, e.g., a client computer having a graphical user interface ora Web browser through which a user can interact with an implementationof the subject matter described in this specification, or anycombination of one or more such back-end, middleware, or front-endcomponents.

The components of the system can be interconnected by any form or mediumof digital data communication, e.g., a communication network. Examplesof communication networks include a local area network (“LAN”) and awide area network (“WAN”), an inter-network (e.g., the Internet), andpeer-to-peer networks (e.g., ad hoc peer-to-peer networks).

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of any claims,but rather as descriptions of features specific to particularimplementations. Certain features that are described in thisspecification in the context of separate implementations can also beimplemented in combination in a single implementation. Conversely,various features that are described in the context of a singleimplementation can also be implemented in multiple implementationsseparately or in any suitable subcombination.

Moreover, although features can be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination can be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingcan be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

As such, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking orparallel processing can be utilized.

It is intended that the specification and embodiments be considered asexamples only. Other embodiments of the disclosure will be apparent tothose skilled in the art in view of the specification and drawings ofthe present disclosure. That is, although specific embodiments have beendescribed above in detail, the description is merely for purposes ofillustration. It should be appreciated, therefore, that many aspectsdescribed above are not intended as required or essential elementsunless explicitly stated otherwise.

Various modifications of, and equivalent acts corresponding to, thedisclosed aspects of the example embodiments, in addition to thosedescribed above, can be made by a person of ordinary skill in the art,having the benefit of the present disclosure, without departing from thespirit and scope of the disclosure defined in the following claims, thescope of which is to be accorded the broadest interpretation so as toencompass such modifications and equivalent structures.

It should be understood that “a plurality” or “multiple” as referred toherein means two or more. “And/or,” describing the associationrelationship of the associated objects, indicates that there may bethree relationships, for example, A and/or B may indicate that there arethree cases where A exists separately, A and B exist at the same time,and B exists separately. The character “/” generally indicates that thecontextual objects are in an “or” relationship.

In the present disclosure, a first element being “on” a second elementmay indicate direct contact between the first and second elements,without contact, or indirect geometrical relationship through one ormore intermediate media or layers, unless otherwise explicitly statedand defined. Similarly, a first element being “under,” “underneath” or“beneath” a second element may indicate direct contact between the firstand second elements, without contact, or indirect geometricalrelationship through one or more intermediate media or layers, unlessotherwise explicitly stated and defined.

Some other embodiments of the present disclosure can be available tothose skilled in the art upon consideration of the specification andpractice of the various embodiments disclosed herein. The presentapplication is intended to cover any variations, uses, or adaptations ofthe present disclosure following general principles of the presentdisclosure and include the common general knowledge or conventionaltechnical means in the art without departing from the presentdisclosure. The specification and examples can be shown as illustrativeonly, and the true scope and spirit of the disclosure are indicated bythe following claims.

The invention claimed is:
 1. A method for image capturing, applicable toa terminal, the method comprising: in response to determining that theterminal is in an augmented reality image capturing scene, determiningaugmented reality environment information of the augmented reality imagecapturing scene; and in response to receiving an image capturinginstruction and the image capturing instruction being an instruction forcapturing an image, capturing an image and recording the augmentedreality environment information corresponding to the image, to obtain atarget image carrying the augmented reality environment information,comprising: capturing a preview image; recording the augmented realityenvironment information for acquiring the preview image using anaugmented reality algorithm; associating the preview image with theaugmented reality environment information for acquiring the previewimage, to obtain the target image carrying the augmented realityenvironment information; packaging the augmented reality environmentinformation of the target image into a file of the target image in amanner depending on a file storage type of the target image, to obtain atarget picture carrying the augmented reality environment information;and in response to receiving an instruction for performing augmentedreality on the target picture, performing an augmented realityprocessing on the target picture according to the augmented realityenvironment information of the target picture; wherein packaging theaugmented reality environment information comprises: writing a length ofthe augmented reality environment information into XMP metadata of thefile and attaching the augmented reality environment information to anend of the file in response to the file storage type is JPEG format;wherein performing the augmented reality processing on the targetpicture comprises: in response to the instruction for performingaugmented reality on the target picture, obtaining the length of theaugmented reality environment information from the XMP metadata,obtaining the augmented reality environment information by determining astarting point for the augmented reality environment information basedon the length and an end position of the file, and performing theaugmented reality processing using the augmented reality environmentinformation.
 2. The method of claim 1, wherein the augmented realityenvironment information comprises: one or more of plane information,light source information, and environment depth information of an imagecapturing scene.
 3. The method of claim 1, wherein the image capturinginstruction is an instruction for capturing a video; and capturing theimage and recording the augmented reality environment informationcorresponding to the image, to obtain the target image carrying theaugmented reality environment information comprises: when capturing thevideo, sequentially recording the augmented reality environmentinformation corresponding to each video frame of the shot video using anaugmented reality algorithm; and according to a display timestamp ofeach video frame, associating each video frame of the shot video withthe augmented reality environment information corresponding to eachvideo frame, to obtain a target video carrying the augmented realityenvironment information.
 4. The method of claim 3, further comprising:according to a file storage type of the target video, packaging theaugmented reality environment information of the target video into afile of the target video to obtain a target video file carrying theaugmented reality environment information; and in response to receivingan instruction for performing augmented reality on the target videofile, performing an augmented reality processing on the target videofile according to the augmented reality environment information of thetarget video file.
 5. A mobile terminal implementing the methodaccording to claim 1, wherein the mobile terminal has an image capturingapplication installed therein, and comprises a display screen configureddisplay the captured image through the image capturing application. 6.The mobile terminal according to claim 5, wherein the mobile terminal isconfigured to, according to the AR environment information carried inthe captured image, enable modification and edition of the AR effect onthe captured image without enabling the image capturing application. 7.An apparatus for image capturing, comprising: a processor; a memorycomponent configured to store instructions executable by the processor;wherein the processor is configured to: in response to determining thatthe terminal is in an augmented reality image capturing scene, determineaugmented reality environment information of the augmented reality imagecapturing scene; and in response to receiving an image capturinginstruction and the image capturing instruction being an instruction forcapturing an image, capture an image and record the augmented realityenvironment information corresponding to the image, to obtain a targetimage carrying the augmented reality environment information by:capturing a preview image; recording the augmented reality environmentinformation for acquiring the preview image using an augmented realityalgorithm; associating the preview image with the augmented realityenvironment information for acquiring the preview image, to obtain thetarget image carrying the augmented reality environment information;packaging the augmented reality environment information of the targetimage into a file of the target image in a manner depending on a filestorage type of the target image, to obtain a target picture carryingthe augmented reality environment information, comprising: writing alength of the augmented reality environment information into XMPmetadata of the file and attaching the augmented reality environmentinformation to an end of the file in response to the file storage typeis JPEG format; and in response to receiving an instruction forperforming augmented reality on the target picture, performing anaugmented reality processing on the target picture according to theaugmented reality environment information of the target picture; whereinthe processor is further configured to: in response to the instructionfor performing augmented reality on the target picture, obtain thelength of the augmented reality environment information from the XMPmetadata, obtain the augmented reality environment information bydetermining a starting point for the augmented reality environmentinformation based on the length and an end position of the file, andperform the augmented reality processing using the augmented realityenvironment information.
 8. The apparatus of claim 7, wherein theaugmented reality environment information comprises: one or more ofplane information, light source information, and environment depthinformation of an image capturing scene.
 9. The apparatus of claim 7,wherein the image capturing instruction is an instruction for capturinga video; and the processor is further configured to: when capturing thevideo, sequentially record the augmented reality environment informationcorresponding to each video frame of the shot video using an augmentedreality algorithm; and according to a display timestamp of each videoframe, associate each video frame of the shot video with the augmentedreality environment information corresponding to each video frame, toobtain a target video carrying the augmented reality environmentinformation.
 10. The apparatus of claim 9, wherein the processor isfurther configured to: according to a file storage type of the targetvideo, package the augmented reality environment information of thetarget video into a file of the target video to obtain a target videofile carrying the augmented reality environment information; and inresponse to receiving an instruction for performing augmented reality onthe target video file, perform an augmented reality processing on thetarget video file according to the augmented reality environmentinformation of the target video file.
 11. A non-transitorycomputer-readable storage medium having instructions stored therein,wherein when the instructions are executed by a processor, a method forimage capturing is implemented, the method comprising: in response todetermining that the terminal is in an augmented reality image capturingscene, determining augmented reality environment information of theaugmented reality image capturing scene; and in response to receiving animage capturing instruction and the image capturing instruction being aninstruction for capturing an image, capturing an image and recording theaugmented reality environment information corresponding to the image, toobtain a target image carrying the augmented reality environmentinformation, comprising: capturing a preview image; recording theaugmented reality environment information for acquiring the previewimage using an augmented reality algorithm; associating the previewimage with the augmented reality environment information for acquiringthe preview image, to obtain the target image carrying the augmentedreality environment information; packaging the augmented realityenvironment information of the target image into a file of the targetimage in a manner depending on a file storage type of the target image,to obtain a target picture carrying the augmented reality environmentinformation; and in response to receiving an instruction for performingaugmented reality on the target picture, performing an augmented realityprocessing on the target picture according to the augmented realityenvironment information of the target picture; wherein packaging theaugmented reality environment information comprises: writing a length ofthe augmented reality environment information into XMP metadata of thefile and attaching the augmented reality environment information to anend of the file in response to the file storage type is JPEG format;wherein performing the augmented reality processing on the targetpicture comprises: in response to the instruction for performingaugmented reality on the target picture, obtaining the length of theaugmented reality environment information from the XMP metadata,obtaining the augmented reality environment information by determining astarting point for the augmented reality environment information basedon the length and an end position of the file, and performing theaugmented reality processing using the augmented reality environmentinformation.
 12. The non-transitory computer-readable storage medium ofclaim 11, wherein the augmented reality environment informationcomprises: one or more of plane information, light source information,and environment depth information of an image capturing scene.
 13. Thenon-transitory computer-readable storage medium of claim 11, wherein theimage capturing instruction is an instruction for capturing a video; andcapturing the image and recording the augmented reality environmentinformation corresponding to the image, to obtain the target imagecarrying the augmented reality environment information comprises: whencapturing the video, sequentially recording the augmented realityenvironment information corresponding to each video frame of the shotvideo using an augmented reality algorithm; and according to a displaytimestamp of each video frame, associating each video frame of the shotvideo with the augmented reality environment information correspondingto each video frame, to obtain a target video carrying the augmentedreality environment information.
 14. The non-transitorycomputer-readable storage medium of claim 13, wherein the method furthercomprises: according to a file storage type of the target video,packaging the augmented reality environment information of the targetvideo into a file of the target video to obtain a target video filecarrying the augmented reality environment information; and in responseto receiving an instruction for performing augmented reality on thetarget video file, performing an augmented reality processing on thetarget video file according to the augmented reality environmentinformation of the target video file.